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design  specifications  for  training  simulators  should  include  features  intended  to  promote 
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FOREWORD 


This  paper  is  the  basis  for  an  invited  talk  delivered  23  July  1981  at  a  workshop  on 
"Research  Issues  in  the  Determination  of  Simulator  Fidelity”  held  by  the  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences,  Alexandria,  Virginia.  It  provides  a  review 
and  statement  about  the  current  "state  of  the  art"  in  the  design  of  simulation  for  training. 
It  was  intended  to  raise  questions  regarding  the  specification  of  the  characteristics  of 
simulations  for  training  that  are  in  need  of  research  and  development  attention. 

The  paper  is  the  second  prepared  under  laboratory  independent  exploratory  develop¬ 
ment  project  ZF66-5 12-00 1-0  3.08  (Theory  of  Graphic  Representation).  In  the  project,  a 
preliminary  review  of  the  literature  and  current  practices  in  designing  instruction  and 
textual  communication  revealed  that  no  theoretical  framework  existed  to  serve  as  the 
basis  for  techniques  needed  to  determine  appropriate  graphical  forms  of  presentation. 
The  overall  goals  of  tlie  project  are  to  survey  current  techniques  for  determining 
instructional  representations,  review  the  state  of  current  theory  about  how  people 
represent  knowledge,  and  derive  and  test  prescriptions  for  effective  means  of  represent¬ 
ing  instruction. 

The  project  is  part  of  the  five-year  plan  for  exploratory  development.  The  review 
and  subsequent  empirical  research  effort  are  focused  on  the  role  of  representations  in 
learning  and  how  to  arrange  graphic  instructional  presentations  to  train  efficiently  and 
effectively.  The  theoretical  description  of  learner  knowledge  structures  is  intended  to 
lead  to  the  development  of  techniques  for  specifying  how  to  match  task  requirements  and 
learners'  structures  in  training.  It  will,  therefore,  be  of  ultimate  use  in  training  design 
and  development.  The  empirical  studies  will  develop  directly  usable  training  materials 
because  the  efforts  will  be  carried  out  on  Navy  training  tasks. 


JAMES  F.  KELLY,  JR. 
Commanding  Officer 


JAMES  J.  REGAN 
Technical  Director 
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SUMMARY 


Problem 


The  word  "fidelity,"  when  applied  to  simulators  and  training  devices,  means  different 
things  to  different  people.  At  a  general  level,  it  is  assumed  that  high  fidelity  is  necessary 
for  a  device  to  be  an  effective  trainer.  "High  fidelity"  is  often  defined  in  terms  ol 
whether  the  device  generally  actually  looks,  feels,  smells,  and  runs  like  the  object  being 
simulated.  The  problem  is  that  this  approach  is  too  general.  Specifications  for  the 
training  the  device  is  to  support  are  vague.  The  design  specifications  do  not  specify  the 
actual  training  requirements  to  guide  performance  measurement  to  document  learning  nor 
do  they  specify  the  use  conditions  necessary  for  training.  This  seems  to  be  due  to  a 
limited  "mental  model"  or  conception  of  how  to  specify  tne  design  requirements  held  by 
program  managers  who  are  responsible  for  procuring  simulators  and  training  devices  for 
new  weapon  systems. 

Objectives 

The  objectives  of  this  paper  are  to: 

1.  Introduce  the  concept  of  mental  models  to  provide  perspective  about  specifying 
principles  for  designing  training  devices  and  suggest  why  they  are  not  designed  well. 

2.  Review  the  design  inadequacies  of  planned  Navy  training  device  procurements  as 
evidence  that  tlie  state  of  the  art  is  poor. 

3.  Propose  some  guidelines  that  should  be  included  in  the  design  specifications. 
Approach 

Recent  work  on  the  analysis  of  learners'  representations  of  complex  physical  systems 
was  reviewed.  This  was  used  to  introduce  the  conception  of  mental  models  to  show  that 
they  are  an  important  part  of  what  must  be  considered  in  designing  simulations  and,  if 
faulty,  they  impede  development  of  the  state  of  the  art. 

Findings 

The  review  summarizes  the  characteristics  or  principles  that  should  guide  the 
acquisition  of  training  simulators.  It  is  pointed  out  that,  in  some  cases,  the  requirements 
of  designing  a  good  trainer  may  force  a  departure  from  the  physical  and  operational 
characteristics  of  the  object  or  system  being  simulated.  There  will  be  resistance  to  such 
departure  from  operational  realism.  An  educational  effort  appears  necessary  to  change 
people's  mental  models  so  tha\  the  art  can  be  advanced. 

Conclusions 

Systematic  guidelines  need  to  be  prepared  from  available  research  and  practice  to 
guide  the  design  specification  of  training  devices.  Fidelity  requirements  are  derived 
directly  from  specific  training  objectives  and  conditions  known  to  produce  effective 
training.  They  cannot  be  specified  solely  in  a  general  fashion  based  on  the  physical 
characteristics  of  the  equipment  and  tasks  to  be  accomplished. 
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Recom  mendatiot  > 

A  substantial  effort  is  recommended  to  develop  the  guidelines  for  improving  the 
state-of-the-art  capability  in  designing  training  devices.  In  addition,  systematic  imple¬ 
mentation  of  these  guidelines  is  necessary.  In  some  cases,  research  may  be  necessary  to 
clarify  some  persistent  problems  and  improve  the  design  guidelines. 
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INTRODUCTION 


Objectives 

The  objectives  of  this  paper  are  to: 

1.  Introduce  the  concept  of  mental  models  to  provide  perspective  about  specifying 
principles  for  designing  training  devices  and  suggest  why  they  are  not  designed  well. 

2.  Review  the  design  inadequacies  of  planned  Navy  training  device  procurements  as 
evidence  that  the  state  of  the  art  is  poor. 

3.  Propose  some  guidelines  that  should  be  included  in  the  design  specifications. 
Background 


The  word  "fidelity,"  as  applied  to  simulators  and  training  devices,  means  different 
things  to  different  people.  At  a  general  level,  it  is  assumed  that  high  fidelity  is  necessary 
for  a  device  to  be  an  effective  trainer.  "High  fidelity"  is  often  defined  in  terms  of 
whether  the  device  generally  looks,  feels,  smells,  and  runs  like  the  actual  object  being 
simulated.  The  problem  is  that  this  approach  is  too  general.  The  device  is  to  support 
training  but  training  is  only  vaguely  specified.  The  design  specifications  do  not  include 
the  actual  training  requirements  to  guide  the  performance  measurements  that  document 
learning  nor  do  they  specify  the  conditions  necessary  to  bring  about  training.  This  seems 
to  be  due  to  a  limited  "mental  model"  or  conception  of  how  to  specify  the  design 
requirements.  Program  managers,  who  are  responsible  for  procuring  simulators  or 
training  devices  for  new  weapon  system?,  know  little  about  training  design  principles  and, 
therefore,  do  not  consider  them.  This  paper  discusses  criteria  that  may  be  used  to  specify 
the  degree  and  type  of  fidelity  needed  for  training  using  simulation.  First  is  the  concept 
of  mental  models.  An  analysis  of  the  mental  model  to  be  learned  by  trainees  can  produce 
a  task  representation  (simulation)  that  substantially  differs  from  physical  reality.  Second 
is  a  discussion  of  criteria  associated  with  pedagogical  or  instructional  features.  These 
criteria  must  be  included  in  design  of  training  systems  or  devices. 

The  notion  of  mental  models  is  a  useful  way  to  discuss  some  issues  relevant  to  the 
question  of  the  fidelity  of  simulations  for  effective  training  and  maintenance  of 
competency.  The  mental  models  notion  is  really  not  new,  but  is  receiving  considerable 
recent  attention  among  cognitive  scientists  (e.g.,  3ohnson-Laird,  1980;  de  Kleer  <Jc  Brown, 
1981).  The  notion  has  considerable  heuristic  value  that  can  be  used  in  the  current 
discussion. 

Basically,  the  idea  is  simple:  A  person  makes  use  of  an  internal  model  of  the  world  to 
reason  and  explain  things  about  the  world. 

If  the  organism  carries  a  "small-scale  model"  of  external  reality 
and  of  its  possible  actions  withir  its  head,  it  is  able  to  try  out  various 
alternatives,  conclude  which  is  the  best  of  them,  react  to  future 
situations  before  they  arise,  utilize  the  knowledge  of  past  events  in 
dealing  with  the  present  and  the  future,  and  in  every  way  to  react  in 
a  much  fuller,  safer  and  more  competent  manner  to  the  emergencies 
which  face  it.  (K.  Craik,  1943,  cited  by  lohnson-Laird,  1980.) 

Mental  models  are  composed  of  autonomous  objects  associated  topologically  with  others, 
rules  for  their  interaction  that  allow  them  to  be  "run"  in  one's  imagination  or  mind's  eye 


and  the  outcomes  assessed.  They  assist  human  reasoning  by  producing  explanations  or 
justifications  of  complex  system  behavior.  They  are  mnemonic  devices  or  learning 
devices  (Hollan  &  Williams,  in  preparation).  Thus,  they  aid  in  the  apprehension  of  how 
systems  work  and  provide  a  strong  means  for  generating  expectancies  about  how  things 
are  done,  should  be  done,  or  the  consequences  of  certain  actions.  It  is  important  to 
realize  that  mental  models  are  powerful  analogical  devices  humans  use  in  learning  about 
their  world. 

Tnere  are  two  main  points  to  emphasize  about  this  notion  of  mental  models.  First, 
people  develop  mental  models  through  the  interaction  with  physical  and  social  systems. 
Thus,  we  must  consider  the  best  way  to  present  materials  and  simulations  to  develop 
effective  training.  The  principles  for  designing  and  critiquing  instructional  presentations 
(simulations)  need  to  be  developed  (de  Kleer  &  Brown,  1981).  Second,  since  adults  may 
have  strong  models  replete  with  errors  or  ’hugs,"  they  can  also  inhibit  new  learning,  or 
changes  in  systems.  So,  on  the  one  hand,  mental  models  are  used  by  individuals  to 
consider  systems  qualitatively  and  "handle"  their  world  and,  on  the  other,  they  may  be 
barriers  to  change  or  correction. 

In  training  with  simulation,  we  are  attempting  to  help  people  build  up  their 
representation  of  the  physical  world  to  he  able  to  operate  equipment  effectively;  we  must 
design  training  and  training  devices  to  allow  the  most  effective  buildup  o»  appropriate 
mental  models.  Many  of  tlie  tasks  to  be  trained  are  procedural  and  involve  decisions 
about  reporting  and  classifying  signals,  or  anticipating  outcomes  resulting  from  changes  in 
a  system.  Simulation  for  these  tasks  must  be  dene  in  a  way  that  maps  into  what  the 
trainees  already  understand  as  well  as  the  critical  aspects  of  the  terminal  task  to  be 
learned. 

In  addition  to  needing  to  consider  the  mental  representations  of  the  learner  to  design 
simulators,  one  needs  to  be  concerned  with  the  mental  models  of  training  simulators 
possessed  by  the  managers  of  the  personnel  and  training  system,  the  program,  or  the 
weapons  system.  From  the  point  of  view  of  the  educational/training  psychologist,  these 
people  are  laymen.  They  do  not  consider  how  students  represent  knowledge  and  physical 
systems  to  themselves  or  that  performance  measurement,  practice,  and  feedback  are 
needed  for  learning.  Thus,  it  has  been  difficult  to  introduce  systematic  changes  into  how 
they  specify  requirements  for  training  devices. 

Problem 

Limited  Conception  of  Simulation  for  Training 

When  "fidelity"  of  simulation  is  discussed,  most  people  think  of  how  much  physical 
resemblance  or  isomorphism  exists  between  the  simulated  and  "real"  task  situations.  This 
seems  to  be  the  reason  that  we  work  so  hard  to  create  simulates  that  look,  feel,  smell, 
bounce  around,  and  otherwise  appear  to  operate  like  the  real  equipment.  A  simulated 
plane's  cockpit  must  be  indistinguishable  from  the  real  thing.  A  simulated  power  plant 
must  generate  steam  power.  This  philosophy  guiding  the  construction  of  simulators  is 
costly,  but  is  considered  worthwhile  because  it  is  so  real.  Fidelity  is  a  close  synonym  of 
"realism"  in  this  usage.  Realism  may  be  useful  for  some  things  at  some  times,  but  it  may 
not  provide  the  best  training  situation  in  many— even  most— instances  and  it  may  slow 
learning  down  considerably. 

This  probably  describes  most  laypersons'  idea  or  model  of  what  an  effective  simulator 
or  training  device  should  be  like.  Mental  models  like  this  influence  expectancies  and, 
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therefore,  what  they  buy  or  build  when  they  are  assigned  the  task  of  developing  training 
devices.  Since  most  program  or  training  system  managers  are  ’’laymen"  when  it  comes  to 
knowing  how  to  train,  the  simulation  characteristics  needed  for  effective  and  efficient 
training  are  often  not  specified  and  required.  Similar  perspectives  about  classroom 
instruction  held  by  teachers  or  instructors  allow  ineffective  methods  to  persist  and  slow 
down  the  rate  of  acceptance  of  more  effective  teaching  techniques  (Montague,  Ellis,  it 
Wulfeck,  1981).  Everyone  thinks  they  know  how  to  teach  because  they  went  to  school. 
Similarly,  it  is  easy  to  assume  that  a  realistic  training  device  will  be  an  effective  trainer. 
Such  misapprehensions  are  difficult  to  overcome.  In  these  cases,  the  problem  is  that  the 
training/teaching  is  approached  in  too  general  a  fashion.  No  specification  of  training 
requirements  and  objectives  is  made  with  the  consequential  failure  to  define  performance 
criteria  needed  to  judge  the  success  or  failure  of  trainees. 

Example  of  the  Problem  Showing  the  State  of  the  Art 

Let  us  examine  the  consequences  of  this  limited  mental  model  in  action  by  examining 
the  state  of  the  art  of  designing  simulators.  In  this  case,  state  of  the  art  is  defined  in 
terms  of  tlie  simulators  being  acquired  for  training  sonar  operators  and  scheduled  to  go 
into  use  sometime  in  1985.  We  will  outline  the  task,  briefly  describe  the  training,  and 
discuss  the  concepts  behind  the  simulation  to  be  used  for  training.  From  this,  we  can 
evaluate  the  quality  of  these  training  simulators  based  on  what  is  known  to  be  required  for 
effective  training  and  for  understanding  the  effect  of  more  limited,  erroneous,  but 
prevalent  mental  model  on  training  device  design. 

The  tasks  ol  detecting  and  then  classifying  signals  on  various  displays  are  the  primary 
and  mission-critical  tasks  required  of  sonar  operators-  They  are  complicated  and  difficult 
and,  interestingly,  they  are  not  emphasized  in  training.  Operator  training  concentrates  on 
the  somewhat  simpier  tasks  of  searching,  tracking,  ami  localizing  possible  targets 
(Mackie,  Shultz,  U  Beardon,  1981).  The  trainers  for  operators  historically  use  simplified 
detection  and  classification  tasks,  sometimes  eliminating  them  completely  as  tasks 
performed  in  tiaining.  A  few— rare— operator  trainers  have  been  developed  recently  that 
present  recorded  or  good  simulated  signals  however,  they  do  not  provide  assessment  of 
errors  and  appropriate  feedback.  Thus,  the  use  of  these  devices  in  schoolhouse  training  is 
unlikely  to  train  operators  effectively  to  enable  them  to  perform  well  when  they  are 
assigned  the  tasks  ol  detecting  and  classifying  signals  on  the  job. 

The  job  itself  does  not  provide  systematic  experiences  that  will  develop  the  detection 
and  classification  skills.  At  sea,  most  ships  are  in  areas  that  have  few  contacts  of 
military  significance.  !r.  addition,  a  sonarman  is  assigned  tc  many  tasks  other  than  sonar 
watchstanding.  Even  when  a  ship  regularly  participates  in  exercises,  there  is  no  assurance 
that  each  operator  will  receive  systematic  exposure  to  the  military  targets  so  that  they 
can  develop  the  high  skill  levels  needed  to  perform  the  complex  pattern  recognition  tasks 
involved.  Novice  operators  are  not  assigned  systematically  during  exercises  and  there  is 
not  sound  basis  for  rapid,  thorough  feedback  to  the  operator  regarding  either  successful  or 
erroneous  performance.  Thus,  neither  formal  scho<  ling  nor  on-the-job  experience  provide 
for  the  systematic  development  of  these  skills  so  necessary  to  readiness  and  the  Navy 
mission  (Mackie,  Shultz,  &  Bearden,  1981). 


Naturally,  these  problems  have  not  gone  completely  unrecognized.  However,  it  is 
important  to  examine  the  quality  of  the  solutions  proposed.  By  1985,  the  Navy  is  supposed 
to  deploy  the  AN/SQQ-89  passive  sonar  system  to  many  of  its  ships.  This  system 
integrates  three  sensor  systems:  the  LAMPS  (for  light  airborne  multipurpose  system) 
Mark  III  so  .obuoy  system,  the  tactical  towed  array  system  (TACTAS),  and  the  improved 
hull  mounted  sonar  system.  Because  this  sonar  system  is  so  complex  to  operate,  the  Navy 


is  procuring  a  few,  very  expensive  dynamic  team  trainers  to  provide  teams  with 
"practice."  The  design  concept  for  the  AN/SQQ-89(V)  Acojstic  Operator  Training  System 
is  captured  in  ttw*  words:  "high  fidelity."  The  actual  operational  equipment  is  stimulated, 
giving  the  equipment  the  ability  to  show  targets  as  they  actually  might  occur,  permitting 
tneir  detection  and  classification  and  realistic  execution  of  other  phases  of  ASW 
operations.  On  the  surface,  this  sounds  good,  but  the  cost  restricts  the  number  of  units  to 
be  purchased.  Since  such  simulation  works  in  "real"  time,  few  people  can  be  trained  on 
the  system  and  its  use  is  restricted  to  more  "advanced"  training  in  preference  to  initial 
operator  or  refresher  training.  In  addition,  such  systems  require  many,  highly  trained 
personnel  as  instructors  who  provide  feedback  to  team  members. 

To  take  care  of  more  fundamental  operator  training,  the  Navy  is  developing  a  series 
of  basic  sonar  operator  trainers  (BSOl).  These  are  designed  around  the  mainframe 
cabinets  and  represent  the  operator  consoles  of  specific  sonar  systems.  In  this  instance, 
"high  lideltty"  refers  to  the  physical  consoles  and  their  knobs  and  buttons.  Trainees  can 
manipulate  console  controls  in  response  to  static  information  presented  on  slides  by 
carousel  slide  projectors  holding  80-120  slides.  Also,  they  can  learn  to  set  up  the 
equipment  and  to  perform  certain  procedures  c.<  such  devices.  Training  in  the  critical 
skills  of  detecting  and  classifying  signals  is  rudimentary. 

As  this  example  is  representative  of  the  state  of  the  art  in  simulators  for  training,  we 
can  detect  two  design  approaches.  First,  operational  training  devices  are  designed  to 
provide  high  fidelity  presentation  of  the  actual  equipments  to  be  operated  and  the  signals 
to  be  handled.  These  devices  are  expensive,  work  in  real  time,  and  provide  exercise  for  a 
team  of  operators  on  the  various  suosystems.  This  is  the  case  in  the  example  described, 
as  well  as  in  propulsion  engineering,  aviation,  and  other  complex  weapon  systems.  In 
these  systems,  there  is  no  provision  for  measuring  the  performance  of  various  operators  to 
assess  the  adequacy  oi  their  performance  and  diagnose  then  errors.  There  is  also  no 
provision  for  systematic  exposure  to  critical  or  difficult  signals  and  tasks  appropriate  to 
operator  deficiencies  or  lack  of  skill.  These  systems  make  no  provision  for  rapid  and 
comprehensive  feedback. 

To  take  care  of  basic  operator  training,  the  emphasis  is  on  a  trainer  that  teaches 
people  how  to  set  up  and  calibrate  the  equipment  and  carry  out  some  of  the  simpler 
procedures.  But  the  interactive  capability  is  poor.  The  important  ooerator  tasks  of 
detecting  and  classifying  receive  only  cursory  attention.  The  presentations  are  static  and 
their  number  is  severeiy  limited  by  the  use  of  slides.  The  dynamic  character  of  the  actual 
task  to  be  performed  on  the  job  is  not  well  represented.  In  sonar,  much  of  the  critical 
target  information  is  time  dependent.  Important  aspects  of  performance,  such  as 
detection  latency,  reporting  accuracy,  and  classification  latency  and  errors,  are  not 
addressed.  The  state  of  the  art  in  trainer  design  seems  to  ignore  almost  everything  we 
know  about  how  to  teach  people  to  do  complicated  tasks. 

APPROACH 

The  approach  taken  here  was  to  review  the  characteristics  that  should  be  present  in 
training  devices  and  to  suggest  how  they  might  be  used  prescriptively  in  designing 
simulation. 


FINDINGS  AND  DISCUSSION 


Simulators  that  Teach 

Expensive,  physically  isomorphic  simulators  may  not  be  good  instructional  devices  or 
exercisers  for  a  number  of  reasons.  They  are  so  big  or  complex  that  the  student  cannot 
"see"  the  results  of  his/her  action  on  the  operation  of  the  system.  Events  happen  too 
slowly  over  long  time  periods  to  give  trainees  good  feedback  about  the  consequences  of 
actions.  The  simulators  provide  little  systematic  practice.  The  system  may  not  provide 
good  feedback  about  errors.  In  other  words,  they  do  not  provide  good  analogical  models 
for  learning;  students  cannot  envision  system  functioning.  They  also  violate  what  we 
know  about  how  to  teach  people. 

Training  simulators/devices  are  required  to  provide  the  cues— the  opportunities  for 
trainees  to  respond,  make  and  -correct  errors,  and  observe  the  consequences  of  their 
actions— that  allow  them  to  understand  the  system.  Simulation  that  teaches  well  may 
violate  the  physical  and  temporal  isomorphism,  and  its  effectiveness  in  training  may  be 
due  to  this.  This  idea  is  counterintuitive  to  the  naive  view  that  the  most  important  design 
goal  is  to  provide  high  physical  and  stimulus  fidelity. 

Simulators  that  are  effective  trainers  need  to  simulate  faithfully  the  tasks  to  be 
learned  in  a  way  that  teaches  students  to  think  about  the  problems  they  confront  in  using 
the  equipment  and  to  build  appropriate  mental  representations  of  the  systems  and  the  way 
it  operates.  With  such  simulators,  they  learn  to  operate  a  system  well  and  can  respond 
effectively  to  emergencies  and  anomolies  in  its  operation.  To  do  this,  simulators  need  to 
show— perhaps  quickly,  perhaps  not— what  changes  occur  in  a  system  because  of  certain 
actions.  They  should  allow  frequent  and  rapid  practice  of  the  procedures  to  be  learned*  as 
well  as  allow  and  provide  corrective  feedback  for  errors  so  that  the  errors  can  be 
eliminated.  Often,  the  real  equipment,  as  well  as  physically  and  temporally  close 
simulations  of  the  real  equipment,  do  not  provide  this  instructional  environment. 

Simulators  intended  to  train  novices  and  those  intended  to  provide  extensive  practice 
or  retraining  for  moderately  competent  people  may  need  to  be  designed  quite  differently. 
The  extensive  corrective  feedback  needed  by  these  groups  is  substantially  different. 
Novices  need  extensive  guidance  and  precise  corrective  explanations  for  their  errors  and 
their  causes,  while  already  trained  individuals  may  need  refresher  training  or  to  broaden 
their  knowledge  of  situations,  signals,  or  events  that  may  be  encountered.  They  may  need 
relatively  simplified  examples  of  problems  to  facilitate  learning.  Normally  invisible 
events  or  processes  may  need  to  be  made  visible  to  support  understanding.  More 
experienced  individuals,  on  the  other  hand,  need  to  refine  their  skills,  which  may  require 
more  realistic  representation  of  critical  aspects  of  the  task. 

Primarily,  simulation  needs  instructional  "task  fidelity"  (Semple,  Hennesy,  Sanders, 
Cross,  Beith,  <5c  McCauley,  1981)  and  this  requires  determining  the  training  device  cue  and 
response  capabilities  needed  to  support  learning  and  the  practicing  tasks  that  lead  to  or 
support  the  operational  performance  required.  Instructional  task  fidelity  requires  both 
the  design  of  the  appropriate  represenxation(s)  for  learning  and  inclusion  of  the  necessary 
learning  principles  that  support  acquisition.  A  major  difficulty  is  that  such  design  is  based 
on  learning/cognition  task  analysis  procedures,  which  are  not  well  developed— especially 
for  complicated  tasks  involving  decision  making  and  problem  analysis.  Better  techniques 
are  needed  so  that  training  devices  can  be  designed  to  support  trainee  learning  and 
practice,  rather  than  simply  mimicking  operational  system  characteristics  and  uses.  To 
do  this,  the  final  skills  to  be  learned  need  to  be  specified  in  terms  of  task  performance, 
the  performance  conditions,  and  performance  standards.  Also,  the  steps  involved  in 


apprehending  the  skill  need  to  be  identified,  which  would  include  the  representation(s) 
appropriate  for  the  learner's  competency  level. 

There  is  another  problem  that  impedes  progress  in  the  development  of  specification 
and  standards  for  training  simulators,  especially  since  systematic  performance  assessment 
is  involved.  It  is  resistence  to  being  evaluated.  Users,  who  are  threatened  by  systematic 
measurement  of  performance,  are  often  encountered.  Their  perception  of  the  potential 
for  punishment  when  simulators  can  measure  skill  levels  leads  them  to  avoid  their  use. 
This  creates  problems  in  monitoring  the  use  of  simulators  and  perpetuates  a  mental  model 
of  simulation  that  slows  or  prevents  the  adoption  of  more  effective  principles  of  design. 
Therefore,  systematic  implementation  and  management,  including  educating  the  users  to 
overcome  this  bias,  are  needed. 

The  following  sections  of  the  paper  attempt  to  outline  some  tentative  guidelines  for 
designing  training  devices  in  terms  of  training  task  fidelity.  "Fidelity"  is  directly  linked 
to  the  specification  of  the  knowledge  and  experience  required  to  support  specific 
performance.  The  focus  is  on  identifying  speciiic  training  objectives  and  arranging  the 
process  of  training  to  accomplish  them. 

Some  major  general  goals  cf  training  devices  are  listed  below: 

1.  Build  up  in  the  learner's  mind/ memory  a  representation  of  the  tasks  to  be 
performed  and  how  the  system  functions.  The  goal  is  to  build  the  trainee's  understanding 
of  the  system  and  his  role  in  it  so  that  he  can  make  appropriate  inferences,  under  various 
conditions,  about  what  to  do  and  what  will  happen  as  a  result. 

2.  Produce  smooth,  error-free,  execution  of  needed  procedures/responses. 

3.  Teach  in  a  safe,  less  complicated  environment  and  perhaps  show  the  conse¬ 
quences  of  unsafe  actions. 

4.  Provide  for  the  development  of  skills. 

5.  Provide  tutoring  (especially  intelligent)  that  gives  hints,  coaching,  warnings,  and 
explanations. 

All  simulations  are  trying  to  accomplish  these  goals.  They  try  to  build  up  experience 
thaT  will  aid  the  student  in  the  performance  of  the  real  task.  For  the  most  part,  these 
goals  are  self-explanatory.  The  first  item,  however,  needs  a  little  more  explanation.  A 
major  goal  is  to  help  people  build  a  mental  model  of  the  system  and  its  functioning  and  of 
the  consequences  of  their  actions  in  the  system.  In  complex  systems,  we  would  like  the 
learner  to  develop  an  accurate  understanding  of  the  system  that  will  allow  him  or  her  to 
infer  what  will  happen  if  some  action  is  taken  or  what  produced  some  problem  or 
symptom.  In  part,  limits  in  our  ability  to  specify  and  train  exhaustively  on  complex 
systems  require  this  understanding. 

Guidelines  for  Designing  Simulations  for  Training 

What  makes  a  good  simulation?  There  are  two  main  characteristics.  The  representa¬ 
tions  of  some  aspect  of  objects  or  systems  must  be  recognizable  and  they  must  help  us  to 
understand,  by  manipulating,  enjoying,  or  predicting  their  behavior.  How  can  we 
deliberately  contrive  simulations  to  facilitate  learning  to  provide  insight  into  complex 
system  functioning? 
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To  be  powerful  for  training,  simulators  must  not  only  simulate  reality  and  be  useful  in 
anticipating  and  controlling  that  reality,  but  they  must  affect  or  change  those  manipulat¬ 
ing  them.  Effective  simulators  involve  the  learner;  for  example,  he  perspires  when  he 
"flies"  a  Link  trainer,  he  gets  upset  when  he  views  injustice  in  a  film,  and  he  is  chagrined 
when  he  recognizes  that  he  could  have  caused  a  real  disaster  by  opening  a  valve  too  soon. 

But,  as  just  indicated,  physical  similarity  or  identity  may  not  allow  the  most 
effective  training  (i.e.,  the  most  involvement).  For  example,  Johnson  (1981)  recently 
demonstrated  that  limited  fidelity  simulation  (a  paper  mock-up)  was  more  effective  than 
the  real  equipment  in  producing  performance  resistant  to  forgetting  (over  60  days  or  so). 
This  effect  was  produced  because  the  limited  fidelity  simulation  required  trainees  to  rely 
more  on  their  memory  during  learning.  Similar  findings  have  been  reported  by  a  number 
of  other  researchers  (see  Johnson,  1981,  for  other  citations). 

The  essence  of  simulation  is  placing  the  learner  in  a  sufficiently  realistic  situation 
that  requires  the  same  sequence  of  inquiries,  decisions,  and  actions  as  does  the  actual 
situation.  Each  of  these  activities  triggers  appropriate  feedback  that  may  or  may  not 
alter  the  situation,  but  ihat  can  be  used  for  subsequent  decisions  about  further  actions 
that  may  change  the  nature  of  the  problem.  Therefore,  a  prime  characteristic  of 
effective  simulation  is  how  well  it  mimics  the  critical  characteristics,  problems,  and 
functions  of  the  system  from  the  point  of  view  of  the  learner. 

The  analogical  relationship  between  the  simulation  and  the  real  task  function  is 
primary.  People  build  up  representations  by  observing  the  effects  of  procedures  they 
apply  (at  least  active  practice  is  better,  although  passive  practice  can  be  effective  in 
some  cases).  So  design  must  make  this  observation  possible.  Often,  realistic  simulators 
do  not  make  this  observation  easy:  and  some  even,  impossible  (e.g.,  in  a  steam  plant 
simulator,  the  effects  of  actions  cannot  be  seen  for  long  time  periods  or  without 
physically  moving  somewhere  to  observe  events  or  situation  readouts  on  gauges). 

Teaching  by  simulation  is  teaching  by  analogy.  In  such  teaching,  the  mapping 
between  a  learner's  existing  knowledge  structures  and  the  training  (simulation)  determines 
learning  difficulty  and  how  and  what  is  learned  (Anderson,  Greeno,  Kline,  &  Neves,  1981; 
Bott,  1978;  Rumelhart  <5c  Norman,  1980).  Similarly,  the  mapping  between  the  train¬ 
ing/simulation  and  the  actual  task  should  determine  learning  and  the  adequacy  of  the 
resulting  performance.  Therefore,  "analogical  correspondence"  (Riley,  1981)  between  the 
actual  task  situation  and  the  representation  is  a  primary  dimension  in  designing  simula¬ 
tion.  The  problem,  of  course,  is  defining  that  dimension.  Gentner  (1980)  proposed  a 
structural  characterization  of  good  scientific  analogies  where  the  mapping  between  the 
analogy  and  the  scientific  explanatior./model/task  can  be  specified.  Perhaps  this  method 
can  be  used  to  describe  better  the  relation  between  "simulated"  and  the  "real"  tasks. 

An  analogy  or  simulation  should  be  readily  interpretable  by  trainees.  They  need  to 
understand  how  the  simulated  objects/symbols  represent  the  real  objects.  This  ensures 
the  salience  for  the  learner.  Familiarity  with  the  representational  system  is  thus  an 
important  determiner  of  the  simulation's  adequacy.  It  is  probable  that  this  factor  is  the 
basis  for  requiring  different  types  of  representations  for  people  completely  unfamiliar 
with  the  task  and  system  to  be  learned  than  for  those  needing  refresher  training  or 
extending  their  experience.  When  the  symbols  are  not  understood  (e.g.,  because  of 
cultural  differences)  or  are  otherwise  difficult  to  discriminate,  their  effectiveness  is  poor. 

Another  important  characteristic  is  "transparency."  The  learner  should  be  able  to 
"see"  readily  the  results  of  his  actions  on  the  system  represented.  In  this  sense,  the 
representation  may  be  quite  different  than  the  actual  task.  The  simulation  should  reveal 


the  interaction  of  the  familiar  entities  represented  whether  or  not  they  are  "visib.e" 
normally. 

Salient  design  guidelines  are  listed  below: 

1.  Novice  or  journeyman  level? 

2.  Representation  quality: 

a.  Transparent. 

b.  Understandable  and  familiar. 

c.  Provide  qualitative  understanding. 

3.  Provide  practice.  Cues  and  opportunity  for  error— diagnostic. 

k.  Arranging  training  sequence— relax  certain  constraints,  like  time,  especially 
early  to  produce  better  understanding. 

5.  Systematic  measurement  of  performance— defined  conditions  and  performance 
standards. 

Let's  return  to  the  example  used  for  critiquing  the  state  of  the  art  in  the  design  of  an 
ASW  simulator.  The  example  pointed  out  that  the  simulation  was  "high  fidelity"  in  the 
naive  physical  sense.  The  equipment  and  the  signals  to  be  seen  were  realistic.  However, 
the  training  on  critical  tasks  received  little  emphasis- -performance  and  feedback  about 
errors  was  left  up  to  the  instructors.  What  characteristics  should  be  included  to  design  or 
redesign  the  simulator  to  make  it  an  effective  training  device? 

l.  Fidelity  of  representation.  For  the  novice,  simple  representations  of  problems 
unconfounded  by  multiple  targets  or  poor  transmission  conditions  would  be  needed. 
Perhaps  dynamic  (developing  over  time),  rather  than  static  displays,  would  be  useful  to 
teach  detection  skills.  To  permit  straightforward  transfer  to  the  job,  more  realistic 
displays  could  be  introduced  gradually.  For  more  experienced  individuals,  already 
competent  in  detection  and  tracking,  it  might  be  more  appropriate  to  provide  exposure  to 
a  broad  range  of  likely-to-be-encountered  signals.  For  this,  static  displays  might  be  quite 
appropriate.  What  is  needed  is  a  complete  description  of  what  individuals  do  at  different 
levels  of  competency  and,  then,  more  adequate  design  of  the  simulation  for  promoting 
learning  can  be  contrived. 

2.  Analyze  and  identify  the  critical  aspects  of  the  task.  From  the  analysis,  the 
conditions  and  standards  of  the  performance  measures  that  will  assess  student  progress 
and  evaluate  the  training  are  specified.  Sufficient  opportunity  should  be  provided  for 
making  errors,  especially  common  ones,  and  for  correcting  their  causes.  This  provides 
diagnosis  of  problems  in  student  performance  and  understanding  of  task. 

3.  Vary  problem  development  time  according  to  the  student's  level  of  learning. 
Slow  things  down  for  early  learning  or  to  allow  the  student  to  get  used  to  time  sharing 
among  several  signals  or  tasks.  Change  the  difficulty  systematically  and  progressively  to 
ensure  an  appropriate  range  of  experience.  Adjust  conditions  appropriately  to  maximize 
practice. 

4.  Automate  problem  presentation  and  performance  measurement  to  reduce  in¬ 
structors'  load.  Incorporate  wide  range  >f  signals  to  provide  appropriate  experiences 
needed.  Provide  an  expandible  library  of  signals. 


5.  Provide  lor  record  keeping  and  communication.  Any  system  should  include 
methods  for  keeping  track  of  student  progress  and  providing  informative,  diagnostic 
feedback  to  him/her  about  performance  tests.  In  addition,  instructors  and  managers  need 
information  about  student  progress  and  about  the  adequacy  of  the  course. 

This  brief  listing  of  what  a  simulator  should  include  to  make  it  an  effective  trainer 
needs  to  be  clarified  by  detailed  "how_to"  information  before  it  can  serve  as  a  design 
guide.  Substantial  progress  is  possible  using  available  knowledge.  The  inclusion  of  these 
characteristics  in  simulators  will  do  much  to  make  them  effective  training  devices  and 
provide  a  potential  means  to  assess  competency. 


CONCLUSIONS 

This  paper  began  by  asking;  "Is  simulation  fidelity  the  question?"  In  the  sense  of  a 
naive  realism,  it  is  not.  Realism  may  be  useful  for  some  things  at  some  times,  but  in 
many  instances  it  may  not  provide  the  best  training  situation.  In  fact,  if  it  is  required, 
learning  may  actually  slew  down  considerably.  The  simulation  characteristics  needed  for 
effective  and  efficient  training  are  often  not  present  in  physically  realistic  situations, 
which  is  one  reason  that  some  tasks  hard  to  learn. 

It  is  easy  to  assume  that  a  realistic  training  device  will  be  an  effective  trainer. 
Unless  the  representation  is  easily  understood  by  the  trainee  and  appropriate  training 
conditions  are  included,  however,  this  assumption  is  wrong.  Such  misapprehensions  are 
difficult  to  overcome.  When  this  assumption  is  made,  the  approach  to  the  training/teach¬ 
ing  is  too  general.  As  training  requirements  and  objectives  are  not  specified,  the 
performance  criteria  needed  to  judge  the  success  or  failure  of  trainees  are  not  available. 

To  attain  training  task  fidelity  in  a  simulation,  it  is  necessary  to  describe  the  task 
precisely  in  terms  of  the  way  it  needs  to  be  represented  mentally  and  to  provide 
conditions  for  learning  the  representation.  These  requirements  are  not  met  simply  by 
providing  conditions  very  similar  to  those  of  the  actual  job.  Testing  the  adequacy  of 
performance,  diagnosing  problems  of  understanding  that  lead  to  errors,  and  systematic 
exposure  or  practice  on  infrequent  or  dangerous,  critical  events  are  needed.  The  real 
probiem  is  to  provide  usable  requirements  for  the  simulation  development  that  can  be 
included  as  design  specifications.  Fidelity  requirements  are  derived  directly  from 
students'  cognitive  representations,  specific  training  objectives,  and  conditions  known  to 
produce  effective  training.  They  cannot  be  specified  solely  in  a  general  fashion  based  on 
the  physical  characteristics  of  the  equipment  and  tasks  to  be  accomplished.  Simulation  is 
an  attempt  to  contrive  a  set  of  experiences  for  the  learner  to  allow  learning  to  occur. 
Such  contrivances  should  not  be  designed  casually. 

Perhaps  a  more  substantial  problem  is  to  convince  the  personnel  and  training  system 
managers  and  those  who  develop  new  weapons  systems  to  include  the  guidelines  for  the 
design  of  training  simulation  in  their  specifications.  For  this,  a  political  and  educational 
effort  is  needed.  To  be  effective,  we  need  to  be  persuasive  enough  to  have  training 
characteristics  included  in  specification  standards.  We  have  such  standards  for  instruc¬ 
tional  program  development  (see  Branson,  Rayner,  Cox,  Furman,  King,  &  Harnum,  1975; 
de  Kleer  &  Brown,  1981). 

The  preliminary  set  of  goals  for  simulation  and  the  charcteristics  needed  for  training 
simulation  presented  herein  represent  preliminary  guidelines  for  designing  training  simula¬ 
tions.  Further  elaboration  of  these  characteristics  and  goals,  together  with  the 


development  of  "how-to-do-it"  procedures,  is  needed.  Their  development  will  probably 
improve  the  state  of  the  art  in  designing  effective  simulators  for  ttaining  and  evaluation. 


RECOMMENDATIONS 

Formal  development  of  instructional  prescriptions  for  task  fidelity  should  be  underta¬ 
ken.  This  can  begin  using  existing  knowledge  from  research  and  practice.  However, 
although  there  is  a  growing  body  of  research  analyzing  student  representations  and  their 
role  in  task  performance,  usable  prescriptions  cannot  easily  be  derived  from  this  work. 
Considerable  effort  is  needed  to  refine  the  procedures  for  prescribing  training  and,  even 
though  our  knowledge  is  incomplete,  development  of  design  guidelines  must  begin.  These 
prescriptions  should  then  be  included  in  the  specification  of  all  training  devices.  This  will 
allow  improvement  in  simulator  design  now,  as  well  as  point  out  specific  gaps  in  our 
knowledge  that  additional  research  effort  can  fill.  Finally,  an  implementation  and 
monitoring  system  (quality  control  system)  should  be  formalized  to  ensure  the  proper  use 
of  the  training  devices  and  to  provide  performance  assessment.  Training  for  instructors 
ar.d  training  managers  should  include  a  review  of  the  need  for  systematic  performance 
measurement  to  assess  the  adequacy  of  training  simulation. 
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